[Compilation] Add Unit Tests for VllmFusionPatternMatcherPass#39692
[Compilation] Add Unit Tests for VllmFusionPatternMatcherPass#39692ProExpertProg merged 1 commit intovllm-project:mainfrom
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces unit tests for the VllmFusionPatternMatcherPass, covering pattern registration, UUID stability, and match counting during compilation. A review comment points out that using torch.set_default_device and torch.set_default_dtype modifies global state, which can lead to test flakiness; it suggests using explicit device and dtype placement for models and tensors instead.
| torch.set_default_device("cuda") | ||
| torch.set_default_dtype(torch.float32) | ||
|
|
||
| fusion_pass = ReluFusionPass(vllm_config) | ||
| backend = TestBackend(fusion_pass) | ||
| model = torch.compile(Model(), backend=backend) | ||
|
|
||
| inputs = [torch.rand(8) for _ in range(N)] | ||
| model(*inputs) |
There was a problem hiding this comment.
Using torch.set_default_device("cuda") and torch.set_default_dtype(torch.float32) modifies global state that persists after the test completes. This can lead to unexpected side effects and flakiness in other tests that expect the default CPU device or a different default dtype. It is better to use explicit device and dtype placement for the model and tensors.
| torch.set_default_device("cuda") | |
| torch.set_default_dtype(torch.float32) | |
| fusion_pass = ReluFusionPass(vllm_config) | |
| backend = TestBackend(fusion_pass) | |
| model = torch.compile(Model(), backend=backend) | |
| inputs = [torch.rand(8) for _ in range(N)] | |
| model(*inputs) | |
| fusion_pass = ReluFusionPass(vllm_config) | |
| backend = TestBackend(fusion_pass) | |
| model = torch.compile(Model().to("cuda"), backend=backend) | |
| inputs = [torch.rand(8, device="cuda", dtype=torch.float32) for _ in range(N)] | |
| model(*inputs) |
github-actions
Bot
added
the
ready
…roject#39692) Signed-off-by: BadrBasowid <badr.basowid@gmail.com>
…roject#39692) Signed-off-by: BadrBasowid <badr.basowid@gmail.com>
…roject#39692) Signed-off-by: BadrBasowid <badr.basowid@gmail.com>
…roject#39692) Signed-off-by: BadrBasowid <badr.basowid@gmail.com> Signed-off-by: Avinash Singh <avinashsingh.rcoem@gmail.com>
2 participants
Purpose
Adds unit tests for VllmFusionPatternMatcherPass covering:
uuidis stable across instances with the same pattern types and differs for different types.matched_countandmatch_tablereflect the correct number of matched patterns.Test Plan
Run the tests.
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.